Failed to restore: getCacheEntry failed: connect ETIMEDOUT 20.246.192.124:443

[Leksat]

Why are you starting this discussion?

Bug

What GitHub Actions topic or product is this about?

Actions Cache

Discussion Details

I'm seeing these warnings in FE Build workflow of https://github.com/AmazeeLabs/silverback-template repo:

Warning: Failed to restore: getCacheEntry failed: connect ETIMEDOUT 20.246.192.124:443
Warning: Failed to save: reserveCache failed: connect ETIMEDOUT 20.246.192.124:443

For each warning the timeout is about 5 minutes. Which slows down the workflow significantly.

I checked other workflows/projects using the same workflow step, but everything looks totally fine there. So I guess it's an issue with a particular cache node attached to FE Build workflow on AmazeeLabs/silverback-template.

Here's an example workflow run where the issue exists: https://github.com/AmazeeLabs/silverback-template/actions/runs/15268252233

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[omarabuhussein]

We also have a problem with caching, in our workflow we install the dependencies with Composer , but it is now much slower than before, and these errors can be seen:

Loading composer repositories with package information
:debug::getCacheEntry - Attempt 1 of 2 failed with error: Request timeout: /xedchHqkgo7b4ImCo6DwmNvFRzx5jMbSZ2qv1i124IDH1s8Q8f/_apis/artifactcache/cache?keys=linux-php-8.2.28-lowest-58ca503556e548c5d53652a213e6440e85d5a33cb20c6ab8b2f5b122b2fe8090-%252Clinux-php-8.2.28-lowest-58ca503556e548c5d53652a213e6440e85d5a33cb20c6ab8b2f5b122b2fe8090-%252Clinux-php-8.2.28-lowest-&version=344b5f84623afd0b86cc8e54a5abefa6f0d1d5acdb035bf2f41223ba1dfcf5a5

and

Warning: getCacheEntry failed: Request timeout: /xedchHqkgo7b4ImCo6DwmNvFRzx5jMbSZ2qv1i124IDH1s8Q8f/_apis/artifactcache/cache?keys=linux-php-8.2.28-lowest-58ca503556e548c5d53652a213e6440e85d5a33cb20c6ab8b2f5b122b2fe8090-%2Clinux-php-8.2.28-lowest-58ca503556e548c5d53652a213e6440e85d5a33cb20c6ab8b2f5b122b2fe8090-%2Clinux-php-8.2.28-lowest-&version=344b5f84623afd0b86cc8e54a5abefa6f0d1d5acdb035bf2f41223ba1dfcf5a5

It seems the issue with cache service degradation that happened yesterday is still not completely resolved.

[rjtferreira]

we're also seeing what appears to be some flavor of this issue but on a different IP

Warning: Failed to save: reserveCache failed: connect ETIMEDOUT 52.152.245.137:443

It appears to be something wrong on this IP

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      1 172.19.66.136:40734     52.152.245.137:443      SYN_SENT    -                   
tcp6       0      0 172.19.66.136:33740     140.82.112.22:443       ESTABLISHED -                   
tcp6       0      0 172.19.66.136:37356     20.96.133.71:443        ESTABLISHED -                   
tcp6       0      0 172.19.66.136:37368     20.96.133.71:443        ESTABLISHED -                   
tcp6       0      0 172.19.66.136:52124     20.85.130.105:443       ESTABLISHED -                   
root@actions-runner-no-cache-m6qrz-0:~# telnet 52.152.245.137 443
Trying 52.152.245.137...

[PierreQcMontreal]

Same issue here, Warning: Failed to restore: getCacheEntry failed: Request timeout

[eloyleonardo]

same problem here, the same ip as rjtferreira

Error: getCacheEntry failed: connect ETIMEDOUT 52.152.245.137:443

[samuelcooksey]

also seeing this issue. First noticed it yesterday around 1:00 PM EST
I was able to resolve my issue by updating the versions of some third party actions that I was using.

[nilehmann]

Same here. Just noticed it.

Failed to save: reserveCache failed: connect ETIMEDOUT 20.246.192.124:443

[omarabuhussein]

In our case, the workflows that use this action were failing ramsey/composer-install@v1 , updating this action to v3 ramsey/composer-install@v3 solved the problem, it "might" be because v3 of this action uses actions/cache v4

[nilehmann]

In my case, it was fixed by updating to Swatinem/rust-cache@v2.7.8

[samuelcooksey]

@Leksat in your .github/actions/setup.yml file, try updating the version of HatsuneMiku3939/direnv-action from @v1 to @v1.1.0.
See the change here: https://github.com/HatsuneMiku3939/direnv-action/releases

[Leksat]

Thanks for the suggestion. I tried updating it, but it does not help.

The cache warnings appear in the previous step - jetify-com/devbox-install-action, before HatsuneMiku3939/direnv-action.

Also, they appear only on some projects, but not on others, even if they use exactly same actions.

[PaulBoshoff]

Having the same issue. The longer builds does allow for more coffee breaks though

[klavinsg]

Having the same issue:

Warning: Failed to restore: getCacheEntry failed: connect ETIMEDOUT 52.152.245.137:443

[julik]

Confirmed: we had major issues with caching yesterday. Because we do a dependency install as the first step of most of our workflows, a deploy that takes 10 minutes normally took 60 minutes yesterday. From our perspective the GH artifact cache is down. Is anyone tracking this internally?

[siggerzz]

Same issue in our repo

All dependencies are managed locally by yarn3, the previous cache can be used
Warning: Failed to restore: getCacheEntry failed: connect ETIMEDOUT 20.246.192.124:443

[GhadimiR]

Hi @Leksat,

I'm one of the engineers who works on this service. The tl;dr is that we shut down v3 of the service recently, but only proceeded to terminate the resources hosting the service 2 days ago. This means that what was previously failing quietly (since April) has started failing a bit more loudly, but only for clients attempting to use that older service.

From what I can tell, you're right that the annotations are being produced in jetify-com/devbox-install-action, but more specifically it appears that these annotations are coming from the nix-installer-action, and from what I see, that package specifically does attempt to interact with the decommissioned cache service.

This may mean that action is running a bit slower as it will not be making use of the cache service, but otherwise the only side effect is the annotation you are seeing.

For others seeing this message, please ensure that your packages, and the dependencies of those packages, are all using an up-to-date version of the cache service.

[Leksat]

@GhadimiR Thank you for the reply!

There's a closed issue on nix-installer-action: DeterminateSystems/nix-installer-action#167

Probably jetify-com/devbox-install-action just need to update DeterminateSystems/nix-installer-action to v17. I'll contact them. Thanks again :)

[Leksat]

Reported: jetify-com/devbox-install-action#79

[Dreamsorcerer]

Since about this time, I've been unable to restore caches on v4:

Cache saved with key: yarn-420aa51b4a097ba60fe4c7b5732a41114a4bbcb509b862828f4fd01a6e556cf0-188552fc3fcf9ed13a45613a825b968d14f7585bb2d7cdc81b35ecc270120666
https://github.com/aio-libs/aiohttp-admin/actions/runs/15345637158/job/43181129640?pr=937#step:16:4

Error: Failed to restore cache entry. Exiting as fail-on-cache-miss is set. Input key: yarn-420aa51b4a097ba60fe4c7b5732a41114a4bbcb509b862828f4fd01a6e556cf0-188552fc3fcf9ed13a45613a825b968d14f7585bb2d7cdc81b35ecc270120666
https://github.com/aio-libs/aiohttp-admin/actions/runs/15345637158/job/43181209023?pr=937#step:9:19

[julik]

@GhadimiR I don't know if that helps, but I don't think this is the issue. In our setup, we run some actions on hosted runners and some on our self-hosted runners. We are using the setup-ruby@v1 action which does the caching. On our tasks which run on hosted runners, the cache action apparently does use the correct IPs and all for connecting to the cache service:

Cache key: setup-ruby-bundler-cache-v6-ubuntu-24.04-x64-ruby-3.4.1-wd-/home/runner/work/cheddarsvc/cheddarsvc-with--without--only--v-1-Gemfile.lock-6b31a62eb1d3621ad09599dd8bacc2e3a235571df3e54c42225855ba5620220d
Cache hit for: setup-ruby-bundler-cache-v6-ubuntu-24.04-x64-ruby-3.4.1-wd-/home/runner/work/cheddarsvc/cheddarsvc-with--without--only--v-1-Gemfile.lock-6b31a62eb1d3621ad09599dd8bacc2e3a235571df3e54c42225855ba5620220d
Received 150994944 of 554505691 (27.2%), 142.2 MBs/sec
Received 385875968 of 554505691 (69.6%), 180.8 MBs/sec
Received 554505691 of 554505691 (100.0%), 182.4 MBs/sec
Cache Size: ~529 MB (554505691 B)
/usr/bin/tar -xf /home/runner/work/_temp/4c18dffc-971a-4e2c-b0b4-c1cdadb024ca/cache.tzst -P -C /home/runner/work/cheddarsvc/cheddarsvc --use-compress-program unzstd
Cache restored successfully
Found cache for key: setup-ruby-bundler-cache-v6-ubuntu-24.04-x64-ruby-3.4.1-wd-/home/runner/work/cheddarsvc/cheddarsvc-with--without--only--v-1-Gemfile.lock-6b31a62eb1d3621ad09599dd8bacc2e3a235571df3e54c42225855ba5620220d

But on our self-hosted runners it tries to retrieve them from that box which is not routable. This is not a problem in the self-hosted runner VM as this IP is not routable from any outside networks either (I know we are not restricting access to IPs for our runners):

> bundle install
/opt/hostedtoolcache/Ruby/3.4.1/x64/bin/bundle config --local path /home/ubuntu/runner/_work/cheddarsvc/cheddarsvc/vendor/bundle
/opt/hostedtoolcache/Ruby/3.4.1/x64/bin/bundle config --local deployment true
Cache key: setup-ruby-bundler-cache-v6-ubuntu-22.04-x64-ruby-3.4.1-wd-/home/ubuntu/runner/_work/cheddarsvc/cheddarsvc-with--without--only--v-1-Gemfile.lock-6b31a62eb1d3621ad09599dd8bacc2e3a235571df3e54c42225855ba5620220d
Warning: Failed to restore: getCacheEntry failed: connect ETIMEDOUT 20.246.192.124:443
/opt/hostedtoolcache/Ruby/3.4.1/x64/bin/bundle install --jobs 4
Fetching https://github.com/cheddar-me/customerio-ruby.git
Fetching gem metadata from https://rubygems.org/........
Fetching https://github.com/cheddar-me/customerio-ruby.git
Fetching https://github.com/cheddar-me/fork-blowfish.git
Fetching gem metadata from https://rubygems.pkg.github.com/cheddar-me/..
Fetching rake 13.2.1

> bundle install
/opt/hostedtoolcache/Ruby/3.4.1/x64/bin/bundle config --local path /home/ubuntu/runner/_work/cheddarsvc/cheddarsvc/vendor/bundle
/opt/hostedtoolcache/Ruby/3.4.1/x64/bin/bundle config --local deployment true
Cache key: setup-ruby-bundler-cache-v6-ubuntu-22.04-x64-ruby-3.4.1-wd-/home/ubuntu/runner/_work/cheddarsvc/cheddarsvc-with--without--only--Gemfile.lock-6b31a62eb1d3621ad09599dd8bacc2e3a235571df3e54c42225855ba5620220d
Warning: Failed to restore: getCacheEntry failed: connect ETIMEDOUT 20.246.192.124:443

These logs are from the build of the same SHA, just a few minutes apart. Our runner version on these self-hosted runners is 2.317.0 and the OS is Ubuntu 22. What is going wrong?

[GhadimiR]

So just to be clear, you're actually experiencing the same issue, but a different flavour of it. All of these boil down to "trying to interact with the deprecated service as opposed to the active service". In this instance it's because the version of the runner you're using is pre-cacheV2, and you need to update your runner images to one that is newer than this release. If you're wondering why, it's because the runner is made aware of the newer cache server by the presence of an environment variable that was introduced in that release, hence why GitHub-hosted runners are not encountering a similar issue.

It's rare that we make such breaking changes like this, but generally you should aim to keep your runners up-to-date to avoid issues like this and benefit from any security patches that are getting rolled into these releases.

[julik]

Got it! Yes, updating the runner fixed it.

[theapache64]

Bumping to actions/setup-java@v4 fixed it for me

[joekiller]

Using the following workflow addition before whatever uses the cache seemed to provoke the ACTIONS_CACHE_SERVICE_V2 to work.

    - name: Adding required env vars new cache service
      uses: actions/github-script@v7
      with:
        script: |
          core.exportVariable('ACTIONS_CACHE_SERVICE_V2', process.env['ACTIONS_CACHE_SERVICE_V2']);

[ovcharenko-di]

not in my case, i still get connect ETIMEDOUT even after bumping deps in my custom action to v4